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ABSTRACT 

Description of the characteristics and mechanics of 
the Delphi technique of identifying the likelihood of occurrence of 
specified future events precedes suggested uses for the technique in 
research, planning, and teaching. The procedure involves obtaining 
individual predictions, aggregating results, presenting results to 
the individuals, and having individuals evaluate predictions. A RAND 
Corporation study to predict major developments in population growth, 
probability and prevention of war, future weapon systems, automation, 
space progress and scientific breakthrough illustrates the mechanics 
of the technique. The potential of the procedure for resource 
forecasting, application of the technique to establish city planning 
goals, and a discussion of a modified version of the procedure as 
used in teaching a graduate course in research methodology indicate 
the possible value of the Delphi technique in incorporating variables 
in research and in attempting to identify priorities at various 
spatial scales of planning. (Author/SHM) 
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A problem encountered by investigators in a variety of disciplines is 
that of forecasting future events. Questions concerning the linearity or non- 
linearity and time horizons of variables must be considered, since operation- 
alization requires assumptions about the behaviour of phenomena over time. 
Another difficulty, however, is often that of identifying relevant variables, 
determining a procedure to facilitate their measurement, and incorporating 
them into a forecasting model. For example, when studying land use patterns, 
specific location decisions or consumer behaviour, the impact of transportation 
or marketing innovations could be significant. Or, in analyzing the pattern 
of resource development, the role of variables ranging from new extractive 
processes to national self-consciousness should be considered. These illus- 
trations have a characteristic common to many geographical studies. The 
measurement and incorporation of the identified variables necessitates consid- 
eration of the probability of events occuri.ng in the future. It is the 
aspect of determining how to incorporate variables such as technological 
innovations or social values which prompted this paper. With this consideration, 
the paper examines the utility which a technique developed for futuristic 
forecasting might have for geographical inquiry. The characteristics and 
mechanics of the Delphi technique are briefly described, and then several 
instances in which it might be of use are discussed. It is hoped that the 
potential and demonstrated utility of the technique will result in its 
consideration and eventual application by a wider range of investigators. 

THE DELPHI TECHNIQUE 

The Delphi technique, generally attributed to Helmer, 1 has been described 
as a "...succession of iterative brainstorming rounds..." Briefly, the 
purpose of the technique is to identify the ’lilcel ihood of occurance of specified 
events in the future. The manner in which this is accomplished is by soliciting 
the viewpoints of a number of experts as to whether an event will occur and 
if it will occur as to when its occurance might be expected. The viewpoints 
of individuals are obtained through letter or questionnaire in an attempt to 
minimize the direct impact of the other participating experts upon an indivi- 
dual's judgement. 

The iterative characteristic is attributable to the fact that after 
the viewpoints of individual experts are obtained they are aggregated with 
the viewpoints of other experts. After aggregation, each expert, still in 
isolation from the other participants, is presented with the aggregated results 
and asked if he would alter any of the predictions. In addition, he might 
be asked to state reasons for any viewpoint or prediction with which he 
disagreed. This procedure of obtaining individual predictions, aggregating 
results, presenting them to che individuals, and having individuals evaluate 
the predictions may be conducted for as many rounds as is considered feasible. 




From this brief description, it will appear that the Delphi technique 
belongs to the family of "think-tank" procedures which involve the assembling 
of an array of experts for a given problem. The distinction from other pro- 
cedures, such as group ,r b rains to rming", is that with the Delphi technique 
individuals are ideally kept separate. In this manner, while the individuals 
are influenced by the aggregated results at the end of each round, an attempt 
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is made to decrease the influence of group psychological effects. 

A number of questions obviously require attention in the use of the 
technique. First, the selection of experts or judges may introduce bias in 
the predictions. When considering solutions to environmental problems, for 
example, an engineering-dominated group might be expected to stress development 
of technologically-based procedures while a group of social scientist-dominated 
experts might reach a different prediction set. This problem is vividly 
illustrated in flood plain management where engineers have emphasized structural 
solutions (dams, levies, dikes) while social scientists have urged nons tructural 
approaches (zoning, taxes, insurance). Clearly, for a given problem an attempt 
should be made to assure an appropriate mix of backgrounds among theexperts. 

A second point deserving study is the comparability of predictions made 
by individual experts. When aggregating individual responses, it is essential 
that the collator be able to compare statements from two people in order that 
they may be allocated to appropriate categories. Using environmental problems 
as an example again, one expert might predict a solution from the "development 
of automated waste collectors" while another might predict the "development of 
irrigation systems collecting and utilizing domestic sewage". These two pre- 
dictions represent different levels of generality, and while the latter could 
be taken as a specific illustration of the former it is not difficult to imagine 
responses which would be awkward to compare and evaluate. 

With this very brief introduction to the technique, an example of its 
application by the RAND Corporation may more clearly illustrate the mechanics 
of the procedure. It seems appropriate to use a RAND study since, if Helmer 
may be said to have developed the technique, it appears as if RAND's use of 
the Delphi technique led to its popularization. The technique was used by 
RAND when the corporation was asked to predict the major developments which 
might be expected in six areas: population growth, probability and prevention 

of war, f uture^weapon systems, automation, space progress, and scientific 
breakthroughs . 

As reported by Jantsch, RAND proceeded in the following manner. Eighty- 
two experts from Nortii America and Europe were enlisted and divided into six 
groups, one group for each subject area. In regard to scientific breakthroughs, 
during tiie first round the participants were asked by letter to name breakthroughs 
urgently needed and realizable within the next fifty years. This question 
resulted in forty-nine items being identified. In the second round, the experts 
were asked to estimate when each of the items identified in the first round 
would be realized. Consensus was reached on ten of these items. For the third 
round, the participants were given the ten items for which substantial agreement 
existed, and were asked if those disagreeing with the estimated timing would 
elaborate upon their reasons. As well, seventeen items for which no agreement 
had been readied were presented with the request for experts to explain why 
such a range of viewpoint existed on their timing. In a fourth round, a 
similar procedure was foLlowed as in the preceding round. The final outcome 
was general consensus on the timing of thirty-one of the original forty-nine 
breakthroughs which initially had been identified. Having now considered 
tiie characteristics of the technique, the rest of the paper examines its 
application to problems of geographic interest. 
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POTENTIAL FOR RESOURCE FORECASTING 

It seems that the Delphi technique offers potential for investigations 
of natural resource supply and demand patterns in nations. An integral part 
of sucii Investigations would be a consideration of the importance of changes 
in variables in the future. In an attempt to identify those variables which 
need to be considered in forecasting water requirements, Sewell and Bower have 
provided a list which is relevant to general resource forecasting situations. 

The variables - population, nature of the economy, technology, social tastes, 
policy decisions, nature of the physical system - are similar in that to 
varying degrees it is difficult to estimate the magnitude and timing of changes 
associated with them. For example, technological innovation could result in 
new products by substituting one raw commodity for another. The competition 
between aluminum and wood in houso-b uilding , plastic and glass for containers, 
and petroleum and coal for fuel come to mind. New processes which utilized 
greater or smaller amounts of water or energy to produce the finished product 
could also effectsupply schedules . On the other hand, demand schedules could 
be influenced by social tastes, witness the demand for green lawns and coloured 
paper towels with their associated effect upon water supply requirements. 
Nevertheless, identifying these variables is only one step forward. Having 
recognized their importance, it then becomes necessary to determine how they 
might be incorporated into forecasts. 

A more detailed discussion of technology illustrates the nature of 
the problem. As Sewell and Bower note 

Forecasts of future water demands, particularly where 
industrial demands are significant, need to take into account 
possible technological changes. This is not an easy task. 

First, it is difficult to forecast what technological changes 
* are likely to occur, particularly when the time horizon 
extends beyond five to ten years. Second, it does not follow 
that, because improved technology is developed, it will 
begin immediately. Typically, technological changes find 
their way into practice only slowly, even in a technologically- 
oriented society such as North America. 

Several problems are thus clearly identified in regard to technology: the 

need to identify breakthroughs, the need to estimate the timing of their 
development, the need to estimate the timing of their implementation. These 
aspects arc of common concern to forecasts relating to a range of resource 
management issues. 

A number of alternative approaches are available to accommodate such 
variables as technology. One approach is exemplified by the work of Landsberg 
and colleagues in estimating the adequacy of natural resources in the United 
States to the year 2000. In that study, "reasonably possible estimates in 
technology" were taken into account whenever their form could be envisaged 
and the likely consequences estimated by the investigators. On the basis 
of estimated gains in technology, and the influence of other variables, a 
series of projections were developed to represent low, medium and high patterns 
of resource availability. 

A somewhat different approach which assumes no changes in technology 
ma; be adopted. To some extent this approach was followed by the National 
Energy Board when forecasting supply and demand of Canadian energy from 1966 o 



to 1990. In this study demand was estimated for market sectors, while 
supply was estimated for petroleum, natural gas, coal and electricity. A 
number of assumptions were made as to political, economic, technological and 
social variables over the forecast period. Under technological considerations, 
it was assumed that there would be no new energy form, no new means of 
energy transportation, increased use of electricity and natural gas, and 
increased pressure for the reduction of air and water pollution. In contrast 
with the previous study, it can be seen that little effort was made to estimate 
pr incorporate the impact of changing technology by the National Energy Board. 
This may well be a significant weakness, especially since the extent to which 
changing technology can interact with other variables to make energy projections 
go astray has been documented. In summary, therefore, it would appear as if 
it is inadequate to hold technology constant. On the other hand, neither 
does it appear ideal to include only technological changes when their form 
can be estimated by the investigators involved. 

It appears as if the Delphi technique, as applied by the RAND 
Corporation to estimate scientific breakthroughs, offers considerable scope 
for incorporating some of the variables identified by Sewell and Bower.* In 
particular, the technique seems relevant for estimating the development, timing 
and occurance of technological gains, as well as changes in the economy, social 
tastes and policy decisions. If a pool of experts from government, private 
enterprise and universities, representing a range of disciplinary backgrounds, 
could be assembled, it might be feasible to attempt application of the Delphi 
technique in resource forecasts. In a pilot study, the experts could be divided 
into a number of panels to concentrate upon the variables noted above. If 
the pilot study proved workable, the technique might then be applied to 
individual resource commodities or combinations of commodities at different 
spatial scales over varying time periods. In short, the application of the 
Delphi technique seems to provide a chance to refine and improve forecasting 
methodologies for which present assumptions often seem inadequate. Whether 
done at a county or a national scale, however, the task would be complex and 
demanding. It is with this in mind that the following examples are presented 
to illustrate that the technique may be applied in situations offering more 
immediate dividends. 



APPLICATION IN PLANNING 

The Delphi technique was applied by Brian Turnbull, as Planning Director 
for the City of Waterloo, to determine priorities for area planning goals. 

The City of Waterloo was given a list of twelve planning goals by the Waterloo 
Area Planning Board for consideration. Turnbull felt it was important that 
city aldermen and planning board members discuss the goals in some detail since 
all were desirable and all were phrased in fairly general terms. 

In October 1969, the city aldermen and planning board members were asked 
to evaluate each goal, and assign a number from one to ten to it, with ten being 
the highest possible rating. At a subsequent meeting, the participants were 
presented with the aggregate scores, and discussion ensued. Following the 
meeting, participants were asked to re-score the goals to see if any changes 
had occured as a result of members having encountered other viewpoints. In 
fact, between the two rounds there was no change in the ranking of goals. The 
final priorities for the twelve goals, and their respective scores, were as 
follows: first , to minimize air and water pollution (102); second , development 
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consistent with social and housing needs (82) ; third , to minimize cost of 
essential public services (77); fourth , to preserve the unique attributes of 
the area landscape (72), fourth. to minimize commuting distance and time (72)* 
lixfo, to minimize the urban use of productive agricultural land (67); seventh, 
to develop with respect to requirements of major economic activities 65T 
eig it i , to acilitate and maintain a pattern of local communities (61)* ninth 
to consider economic and technical change and their social implications 7 n 
development (60); jds nth , to maximize opportunities for using specialized 

innn 1C r? (56) ; eleventh, development consistent with technological 

innovation (46) ; and twelfth, to minimize cost of moving goods within the 
region (43) . L1 "- 

. ? he ald ^ rmen and planning board members, taken as separate groups, ranked 
the twelve goals in a similar manner. A notable difference was "to minimize 
-entia public services" (aldermen, second; planning board members, 
ln ^resting similarities were: "to mihimize air and water pollution" 

(first for both groups), to preserve the unique attributes of the area 
landscape (third for planning board members; fourth for aldermen), and 
to minimize cost of moving goods within the region" (eleventh by aldermen- 
tweifth by planning board members). Of additional interest is the fact that 
when Turnbull went through the same procedure with senior geography under- 
graduates at Waterloo Lutheran University the ranking of goals proved to be 
the same * r 

When considering the benefits of the Delphi technique in the modified 
version described above, Turnbull noted several advantages. It forced the 
partrerpants to evaluate a set of goals, alL of which were socially desirable 

^ susce P tibl * to misinterpretation. During the discussion 
between the two rounds, participants found that different rankings by individuals 
often aiose because of dirferent interpretations of the same goal. Having 
ranked the goals and discussed each individually, participants frequently were 
able to resolve differences arising from differing interpretations and to 
put the goals into terms in which they could he clearly understood by laymen. 

Thus, in a practical planning situation, the Delphi technique provided a 
method for a city to systematically evaluate and establish priorities for a 
set of goals. The satisfaction with the procedure is reflected by the grouo's 

'*« technique be utilised by area planning boards or councils 
faced with similar evaluation exercises and Turnbull's intention to use it in 
planning studies with which lie is currently involved in Oxford County. 

APPLICATION IN TEACHING 

A somewhat different modified version of the Delphi technique has 
been used by the writer in graduate seminars. At the University of Waterloo, 
a i 1 . f * r . . yCa f graduate students take a course in research methodology, part 
° involves faculty presenting seminars on selected topics. In the autumn 

of 19 0 a seminar was to focus upon the "Uses and Abuses of Quantification" 

In pr paration for the seminar, each student was asked to indicate what were 
to him the three most significant abuses of quantification in geographical 
research. Tne twenty participating students each submitted a list of abuses, 
wuch was then aggregated wi th the other submissions to identify possible 
patterns. In the seminar itself, the aggregated responses were presented 
to the students under three categories: philosophical, conceptual and 

operational. The abuses noted by the students falling under each of these 
headings .hen served as departure points for discussion. At the conclusion 
of the seminar, the students submitted a new list of the three most significant 
abuses, which in effect constituted a second round in the Delphi procedure. 
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Issue 


Percentages 
Round 1 


of Responses 
Round 2 


Philosophical 

1. Quantification for its own sake, confusing ends 
with means 


ii 


15 


2. Quantification equated with theory, treated as a 
panacea for lack of theory 


5 


6 


3. Confusing definition of quantification: mathematics? 
statistics? logic? all? 


4 


3 


4. Emphasis on math to neglect of geography 


4 


3 


5. Quantification of the trivial, the obvious 
relationship 


2 




6. Manipulation of data to confirm pre-conceived ideas 


2 


— 


Conceptual 

1. Used instead of theory to organize and select research 
problem 


2 




2. Quantifying when too many variables are involved 


2 


- 


Operational 

1. Failure to meet test assumptions 


20 


28 


2. Incorrect interpretation of results 


15 


16 ' 


3. Faulty or inadequate data 


11 


8 


4. Using inappropriate techniques 


8 


5 


5. Faulty sampling procedures 


6 


3 


6. Drawing inferences from descriptive techniques 


4 




7. Underuse: not considering power, power efficiency, 

not combining techniques 


2 


8 


8. Overuse: using techniques which duplicate one 

another 


2 


5 
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The purpose of asking for a second submission was to allow each student to see if 
his own viewpoints had changed, and to allow the writer to determine if any shifts 
in aggregate pattern had occurred. Twelve students responded in the second round, 
and the results of responses to both rounds are presented In Table 1. 

• 4J T ^ G a PP roach described above deviated from the Delphi technique in that 
individuals were not kept separate from one another, but rather discussed the 
aggregate results as a group. In this way, the procedure followed was s imi lar to 
that adopted by Turnbull. On the other hand, each participant had the opportunity 
to evaluate his own priorities before and after the discussion. What were the ' 

benefits of such an approach? First, basing discussion on students' submissions 
kept the seminar relevant to each individual whether or not he was sophisticated 
in his knowledge of quantitative methods. Since each participant had enumerated 
k 8nif leant , abuses prior to the discussion, he was placed in a position where he 
had to defend his observations or be willing to discuss their modification. Second 
the categorization of responses into three classes raised the question as to 
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abuses^'^In'this lUa^fiVt^ d^" ldentifyln 8 

address themselves to Issues at the same level of generaU v f'TT,'" C ° Uld 
effect was the questioning of the relevance or neefT A valuable side 

of abuses. Some wondered whether it wa!! r r !° r 3 classifi cation 

as one of a continuum, ranging ^ ‘ ,r ° ble " 
treme to operational abuses at the other. cai « ethical issues at one ex- 

second round ttcr! ™,°f sUght^hUt tU ° r ° u " ds? Ip ‘«» 

of an operational nature. Some students arumd ^ 1 Lnsoph lcal questions to those 
ferentiate between basic abuse” 1 s“ tol ^ 2™“ n 

there were not problems as long as researchers WP r! f '-i suggested that 

mechanics and implications of Analytical tAchniquAA with assumptions, 

researcher were not familiar with a h Pr i, n . Clnlc ^ es - Problems only arose if the 
of its requirements, or if assumptions were^Vh then unknowingly violated some 
nation, it was felt that these were abuses whi!? 613 ^! i8nored * In either sit- 
line of argument, discussion then shifted to such 0 ° resolvec1, From this 

techniques. In other words, some argued^hat it w 1TI ^ thC underuse of 

techniques without adequate attention to u abuse to use statistical 

efficiency, students ?.lt tCtTese n p ** <1°^ “ d ^ 
their consideration would lead to a ? ° f 4 ten i 8 nored * and that 

Discussion also covered such topic A ^moJiJubll aPP \ 1Cat ° n of techniques, 
about individuals from aggregatA walls 7s l«tl ! UnitS ’ drawlng inferences 
techniques, meeting norm-’litv and ho ^ a using combinations of analytical 
metric or nonparametric techniques. mOSCedastlcit y assumptions, and using para- 

explore a wide range of 8 iAAAes°that t hAd a ioAA°d lem hl thc students were able to 
research endeavours. While after two relevance to their own 

group consensus had not been achieved the i aad dls n ussi °n it was clear that 
ideas concerning posJJflo Si* individuals had sharpened their 

perhaps had had "some of their own questiASrA^Aed^r 011 * ° f problems ’ and 
during thu second round nf iho n«i i • answered. A comment received 

of the session ^ — — - 

I believe (contrary to what I may have said earlier!) that 
- lid in terms of "operational" ’ 

„? C . ts ; Why discuss the use or non-use of quantitative 
Uthods. Obviously they are a necessary tool. Those of 

f eel * th a t G t he re f i <s 1 ^ f,° nversant with these tools and may 

a ^ ua ntification for its own sake", 

this r 3 t- C -° Ur T n PUrP ° Se by dwel ling excessively on 
. , * Cllt i cisms should be confined to how and when 

chniques are used, thus reducing the artificial level 

of qu^tifiAAtlAn'° thiS at ^ m ° lnent overworked topic 

applicat ions e of m the S Delphi n technique| S in a i t s ideal^or^^odif ^d 881 " ^ 

several rounds, could identify problems nr’- TlC Students themselves, through 
could be investigated bv field L bl ?" lss ues m their own community which 
students “P- outcoL, the 

degradation, or upon a range of isrue<! f if \ C lssue » such a s environmental 
poverty and health. In a different co^te^t the en " iron "" ! " t ■ Mutation,- 
to stimulate seminar or class d^trLtT , P rocedu te could be adapted 

Waterloo. m eithtr A? tl dlscuss J on * as w ^ s d °ne at the University of 
either of these types of applications, the technique seems to 
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generate student interest am. involvement in addition to assimilation of 
knowledge . 



CONCLUSIONS 




This paper has attempted to indicate the value, both demonstrated and 
potential., of the Delphi technique for purposes of research, planning and teaching. 
While not a panacea for problems encountered in research involved with forecasting, 
the technique seems to offer one method of incorporating variables which are 
handled inadequately at the moment. At a different level, the technique offers 
interesting applications in teaching, regardless of topic or subject matter, or in 
attempting to identify priorities at various spatial scales of planning. In 

hu’ 4 " ould ap P ear as if more attention might be given co this technique, and 
other similar ones such as the Scenario, as valuable aids for teaching and 
research. 
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